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1. Introduction 



The purpose of this paper is to show a connection between Stanley symmetric 
functions and the formula for quiver varieties given in [Q. Recall that a simple 
reflection in the symmetric group Sm is a transposition that interchanges two con- 
secutive integers. A reduced word for a permutation w G Sm is a tuple of simple 
reflections (ti,T2, . . . ,ti) with £ = £(w) the length of w, such that w = T1T2 ■ • • ti. 
Stanley asked how many reduced words does a permutation w have. 

To answer this question, Stanley defined a power series Fw{x) in infinitely 
many variables xi, 2:2, . . . ; it is homogeneous of degree £ = £{'w), has non- negative 
integer coefficients, and the number of reduced words for w is the coefficient in 
Fw(x) of the monomial X1X2 ■ ■ -xi. Stanley then proved that this power series is 
symmetric. This implies that it can be written in the basis of Schur functions: 

(I) Fi^{x) ^ a^x sx{x) 

where the sum is over all partitions A of £ and the coefficients a^x are integers. 
Since the coefficient of xiX2 • • • in a Schur function sx{x) is equal to the number 
of standard Young tableaux of shape A (see e.g. [|l^ or jf^), it follows that the 
number of reduced words for w is given as 



aw\ f 



The constants are considered well understood, so only a description of the coef- 
ficients Uuix remained to be found. Stanley credits Edelman and Greene for proving 
that these coefficients are non- negative Q (see also |T^). Fomin and Greene have 
shown that a^x is equal to the number of semistandard Young tableaux T of shape 
A, such that the column word of T is a reduced word for w [|7| . Another useful fact 
is that a^-ix — ctw\' where A' is the conjugate of A 0, |l6| , (7.22)]. 

Stanley's symmetric function is known to be a limit of Schubert polynomials 
6w{x) defined by Lascoux and Schiitzenberger jl^, For n G N, let 1" x w e 
Sn+m denote the shifted permutation which acts as the identity on 1 , . . . , n and 
maps i to w{i — n) + n for n+l<i<n + m. If one specializes to finitely many 
variables xi, X2, ■ ■ ■ , xn, then 

(2) Ft„(a;i, . . . ,xjv,0, 0, . . .) = &l,^y^^-l{xl, . . ., a; at, 0,0, . . . ) 

for aU sufficiently large n (7.18)]. 
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The formula for quiver varieties given in |^ specializes to a formula for the double 
Schubert polynomial &io{x;y) for the permutation w G Sm'- 

(3) S^,(a;; y) = ^ c^(a, b, A) ' ' ' VnT-i (-2^2)''^ ■ ■ ■ sx{x/y) . 

The sum is over exponents 02, . . . , a„i_i and 62, ... , fom-i and a partition A. The 
coefficients c^(a, 6, A) are special cases of a large class of generalized Littlewood- 
Richardson coefficients, which are conjectured to be non-negative and given by a 
generalized Littlewood-Richardson rule Q . 

The main result in this paper is that the coefficient Cui(0,0,A) (corresponding 
to zero exponents) is equal to Stanley's coefficient ayj-i\. In this way, (|) writes 
a Schubert polynomial G^ix) as a symmetric polynomial equal to Stanley's sym- 
metric function for plus a non-symmetric polynomial. 

In Section ^ and Section || we review the results of ||] . In Section ^ we prove the 
identity a^j-iA = Cu,(0, 0, A) and use this to give a new proof of Stanley's result 
that the symmetric function F^jg (x) for the longest permutation wq in Sm is equal 
to the Schur function sx{x) for the staircase partition A = (to — 1, to — 2, . . . , 1). In 
Section ^ we use geometry of degeneracy loci to prove a generalization of the well 
known formula for a Schubert polynomial of a product of two permutations. Finally, 
in Section ^, we discuss the relations to the conjectured generalized Littlewood- 
Richardson rule in 

We thank S. Fomin, W. Fulton, and F. Sottile for helpful discussions. 



2. A FORMULA FOR QUIVER VARIETIES 

Let X be a non-singular complex variety and Eq ^ Ei ^ E2 ^ • ■ ■ —>■ En a 
sequence of vector bundles and vector bundle maps over X. A set of rank conditions 
for this sequence is a collection r — (rij) of non-negative integers, for < i < j < n. 
Let nr{E,) C X be the locus where each composite map Ei — > Ej has rank at most 
rij'- 

n^iE.) ^ {x e X \ rank(£'j(a;) ^ Ej{x)) < nj \li < j} . 

This locus determines a cohomology class [Vlr{E,)] in the cohomology ring H*{X) — 
H*{X;Z) of A. 

For convenience we set ra = rank(£'i). We will require that the rank conditions 
r = (vij) can occur, i.e. there should exist a sequence of vector spaces and linear 
maps Vq —>■ Vi ...—!■ T/„ such that dim(l^) = ru and rank(l^ ~^ ^j) — ^ij for 
all i < j. This is equivalent to demanding rij < min(rij_i, r^+ij) for i < j and 
^ij - "^tj-i - + > for j - i > 2 @]. 

If the locus flr{E,) is not empty, its codimension is at most 

dir) = ^(r,;j_i - r,j){r,+ij - r,j) . 

i<j 

Furthermore, this codimension is obtained for generic choices of bundle maps Ei 
Ei^i [||, The main result of ||] gives a formula for the cohomology class 

of the degeneracy locus il.r{E,), when it has this expected codimension d{r). 

To explain this formula, we need some notation. Let A = Z[/ii, /12, . . . ] be the 
ring of symmetric functions. The variable hi can be identified with the complete 
symmetric function of degree i in variables Xi, i £ N. If / = (oi, 02, ... , Op) is a 
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sequence of integers, define the Schur function s/ G A to be the determinant of the 
p X p matrix whose (i, j)"^ entry is ha^+j-i'. 

Si = <i<2t{hai+j-i)l<ij<p ■ 

(Here one sets Hq = 1 and h-q = for g > 0.) A Schur function is always equal to 
either zero, or plus or minus a Schur function for a partition A: 

si = 

This follows from interchanging the rows of the matrix defining sj. Furthermore, 
the Schur functions given by partitions form a basis for the ring of symmetric 
functions (l5|, 

If _B is a vector bundle over X, let c^{E) e denote its i"^ Chcrn class. 

Given two vector bundles E and F of ranks e and / over X one can define a ring 
homomorphism 

A^H*{X) 

which maps hi to the coefficient of in the formal power series expansion of the 
quotient 

ct{F^) 1 - ci{F)t + ■■■ + {-l)fcf{F)tf ■ 

We let s\{F — E) e H*{X) denote the image of sx by this map. (If E and F have 
Chern roots xi, . . . ,Xe and . . . , j// respectively, the notation s\{y/x) = sx{F—E) 
is also common.) 

When the degeneracy locus il.r{E,) has its expected codimension d{r), its coho- 
mology class is equal to a linear combination of products of Schur polynomials in 
differences of consecutive bundles: 

[^r{E.)]=Y.Cf,{r) Sf,,{Ei ~ En)Sf,,{E2 ~ El) ■ ■ ■ Sf,,^{E„ - E„-i) . 

Here the sum is over sequences of partitions fi — (fii,fi2, ■ ■ ■ The coefficients 

c^(r) are integer constants depending on the rank conditions and the sequence fi. 
They are determined by a combinatorial algorithm which we will describe next. 
Start by arranging the rank conditions r — (vij ) in a rank diagram: 

Ef) ^ El ^ E2 ^ • ■ • — > En 

Too rii r22 ■ ■■ rnn 

roi ri2 ■ ■ ■ rn-i,n 

r02 ■ ■ ■ rn-2,n 



ron 

In this diagram, replace each small triangle of numbers 

by a rectangle Rij with r^+ij — rij rows and — columns. 
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These rectangles are arranged in a rectangle diagram: 



Ro2 



Rn 



Rn-2,n 



Ran 

The information carried by the rank conditions is very well represented in this 
diagram. First, the expected codimension d{r) for the locus ilr{E.) is equal to the 
total number of boxes in the rectangle diagram. Furthermore, the condition that the 
rank conditions can occur is equivalent to saying that the rectangles get narrower 
when one travels south-west, while they get shorter when one travels south-east. 
Finally, the algorithm that computes the coefficients c^{r) depends only on the 
rectangle diagram. 

We will define this algorithm by constructing an element P,. in the n^^ tensor 
power of the ring of symmetric functions A®" , so that 



This is done by induction on n. When n = 1 (corresponding to a sequence of two 
vector bundles), the rectangle diagram has only one rectangle R = i?oi- In this 
case we set 

Pr = sn€ A®i , 

where R is identified with the partition for which it is the Young diagram. This 
case recovers the Giambelli-Thom-Porteous formula. 

If n > 2, we let r denote the bottom n rows of the rank diagram. Then r is a 
valid set of rank conditions, so by induction we can assume that 



(4) 



Pr 



is a well defined element of . 
basis element s„, lE" • • • ® s„. 



. Now is obtained from Pf by replacing each 
in (0) with the sum 



E n 



r 1 , . . . , 




Ri-l,i CTi 

n-1 ^ 



Rn 



This sum is over all partitions cti, . . . , cr„_i and ri, . . . , t„_i such that ai has fewer 
rows than and each Littlewood-Richardson coefficient cj^'^. is non-zero. A 

diagram consisting of a rectangle Ri-i^i with (the Young diagram of) a partition 
(Ti attached to its right side, and Ti_i attached beneath should be interpreted as 
the sequence of integers giving the number of boxes in each row of this diagram. 

It can happen that the rectangle Ri-i^i is empty, since the number of rows or 
columns can be zero. If the number of rows is zero, then ai is required to be empty, 
and the diagram is the Young diagram of Ti_i. If the number of columns is zero, 
then the algorithm requires that the length of ai is at most equal to the number 
of rows Tii — ri_i^i, and the diagram consists of di in the top ru — ri_i.i rows and 
Ti-i below this, possibly with some zero-length rows in between. 
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3. Schubert polynomials 
Let w e Sm+i be a permutation, and let E, be a sequence of bundles over X 

Fi C F2 C ■ ■ ■ C Fm Gm Gm-1 ■ ■ ■ ^ Gl 

consisting of a full flag with a general map to a dual full flag. Define the locus 

rt^ ^ {x £ X \ rank(F^(a;) Gp{x)) < r^{p, q) Vp, q} 

where r^{p,q) = #{i < p \ w{i) < q}. Fulton has proved [|| that the cohomology 
class of this locus is given by the double Schubert polynomial defined by Lascoux 
and Schiitzenberger |13|] : 

where Xi = ci(ker(G'i — > Gi-i)) and yi = ci{Fi/ Fi^i). Now fl^j ~ f7.r(i?,) where 
r — (rij) are the obvious rank conditions. This means that the double Schubert 
polynomial becomes a special case of the quiver formula: 

e^{x;y) = [nr{E.)] 

= ^Ci_,{r) S^j(i^2 - Fi)-- ■S^^_^{Fra - Fm-l) ' S^^{Gyn - Fm) ■ 
•^Mm+l {Gm-1 ~ Gm) • ■ ■ (^1 ~ ^2) 

As noted in |5|, significant simplifications can be made by using the equalities 



s\{Fi+i - F,) = sxivi+i) 

and 

sx{Gi - Gi+i) = sx{0/xi+i) = • 



yf^i if A = (a) is a row with a boxes 
otherwise 



{—Xi^i)'' if A = (1'') is a column with b boxes 
otherwise. 



Using this and the fact that sx(Gm — Fm) is the super-symmetric Schur polynomial 
sx{x/y) in the variables xi, . . . , Xm and yi, . . . , ym, we obtain a formula 

(5) 6„(x; y)^Y. ^-(«' ^' ^) ' ' ' ("^2)'^ • • • i^Xm)""' sx{x/y) . 

The sum is over exponents a2, . . . , cim and 62, . . . , 6m, and a single partition A, and 
Cm (a, 6, A) is the coefficient c^ (r) for the sequence of partitions 

M = ((a2),...,(a,„),A,(l^-),---,(l'^))- 

Example 1. For the permutation = 2 43 1 we get the rank diagram 

Fi c F2 C — > G3 — » G2 Gi 

1 2 3 3 2 1 

1 2 2 2 1 

1111 
1 1 

1 
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which in turn gives the rectangle diagram: 

I □ 

□ □ 
□ ■ 



The bottom three rows of this rectangle diagram gives 

P= = (g) 1 (g) 1 ; 



using the algorithm we then get 



and 



Pf = S^(S) s^(g) s^(g)l + 1 (g) sn (g) 1 
□ □ □ □ 1^ 

= (g) (g) (gl(g)l + s^(8)s^(8)s^(g)s^(g)l + 

(g) 1 ® Srjj (g)l®l + S^®l(g)SQ®S^®l + 
1 (gi Sg ® (g)l(gl + l(g)Sg(g)S^(g>S|^(g>l + 

1 (g (g Srjj g)l(g)l + lg)S^g)SQ(giS^(gil + 

□ y-' □ y □ 

1 g) 1 (g) sg^ (glg)l + lg)l(gsgg)s^g)l. 

This gives the formula 

&w{x; y) = 2/2 2/3 s^{x/y) - y2 ys s^{x/y) + y2 s^{x/y) - X3 s^{x/y) 
+ 2/3 s^{x/y) - X3 2/3 s^{x/y) + s^(a;/2/) ~ 2:3 s^ix/y) . 



In general, the rectangle diagram associated to a permutation w € S'm+i contains 
only empty rectangles and 1x1 rectangles, and all of the non-empty ones are located 
in a diamond below the rectangle Rm~i,m- In other words, if Rij is not empty then 
i < m — 1 and j > m. In fact, Rij is non-empty if and only if the diagram 
D'(w) from Q has a box in position (2m — j,i + 1), and this happens exactly when 
w{2m + 1- j) <i + l and w^^{i + 2) < 2m ~ j js). 

4. Stable Schubert polynomials 

In this section we will apply the quiver formula for Schubert polynomials to 
calculate Stanley symmetric functions. Let w G S'm+i be a permutation and r = 
(rij) the corresponding rank conditions. Notice at first that the rank diagram for 
the one step shifted permutation 1 x w is obtained by adding one to each number 
rij in the rank diagram for w, and putting an extra row of ones on the sides of this 
diagram. For example, if w = 3 1 2, this looks like: 

1 2 2 1 1 2 3 3 2 1 

111 1 2 2 2 1 

10 12 11 

111 

1 1 
1 
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This means that the rectangle diagram for 1 x w is obtained by adding a rim of 
empty rectangles to the sides of the rectangle diagram for w. 

□ - I I □ - - 

• □ ^ I • □ - 



Similarly, one obtains the rectangle diagram for 1" x w by adding n rims of empty 
rectangles to the rectangle diagram for w. 

Let Pr £ A®^™"-'^ be the element associated to the rank conditions r = (r^j) for 
w. The above comparison of rectangle diagrams then shows that 1" x w corresponds 
to the element 

1 «) • ■ • O 1 ® 1 «) ■ • ■ «) 1 e A®2'"+2"-^ . 

^ V ' V ' 

n n 

By (|) this gives us 

where sx{x/y) is in variables xi,. .. , x„i+n and yi, . . . , Vm+n- 

Now restrict to two fixed sets of variables xi, . . . ,xn and j/i, . . . ,yM, setting 
Xi — yj — io'c i > N and j > M. When n > max(A^ — 1, M — 1), the only non- 
zero terms in the above expression for &inxw are those with all exponents and 
bi equal to zero. Since this Schubert polynomial is homogeneous of degree equal 
to the length of w, the partitions A occurring in these terms all have weight £{w). 
This proves: 

Theorem. Let w G Sm+i and fix two sets of variables Xi, . . . , xn and yi, . . . , yM- 
When n > max(A^ — l,Af — 1), the double Schubert polynomial Si^xtu 'in these 
variables is given by 

6i"x«,(2;i, ■ • ■ ,a;Ar,0, . . . ,0;yi, . . . ,y7\/,0, . . . ,0) ^ c„(0, 0, A) SA(a;/?/) . 

X\-i(w) 

Comparing with equations (|^) and we obtain (since a^^x — a.^-ix>)'- 

Corollary. Stanley's coefficient a^x is equal to Cu,(0,0, A'). 

Thus the formula (^) writes a Schubert polynomial as a symmetric part equal to 
Stanley's symmetric function plus additional non-symmetric terms. For example, 
if w = 2 4 3 1 as in the above example, we have F^] (x) = ^ ^jn {x) . 

The identity a^y — Q^uj-ia becomes a special case of the identity c^-i (6, a. A') = 
c^(a, b, A), which in turn follows from the formula c^v (r^) = c^('r) of Here 
are the rank conditions obtained by mirroring the rank diagram for r = (Tij) in a 
vertical line, so r^^- = rn-j,n-i, and /i^ is the sequence (/i^, . . . , fi[) of conjugate 
partitions in the opposite order. 

Example 2. Let = ■ ■ • 2 1 be the longest permutation in Sm- Then we have 
r.u]g{p,q) — max(p + q — m,0). The rectangle diagram associated to wq therefore 
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has exactly i non-empty rectangles in the i"^ row for 1 < i < m — 1, and these are 
centered around the middle. All other rectangles are empty. 

I □ - - 
□ □ - }m-l 

□ □ □ 



We will use this diagram to compute Stanley's symmetric function for wq . The idea 

is that in order for the algorithm to produce a contribution to F^^ , all boxes must 
travel north-west until they meet the last non-empty rectangle in this direction, 
and from that point they must travel north-east. 

Let Pr*^^ denote the element in \®'^"i-3-k gjyen by this rectangle diagram with 
the top k rows removed. In particular P^"'' — Pr is the element associated to 
the whole diagram. The terms s\ in Stanley's symmetric function P^jo l"! 
correspondence with terms in Pr of the form 

1 (g) ■ • • (g) l(g)SA g) 1 (g) ■ • ■ (g) 1 . 

^ V ' ^ V ' 

m — 2 m — 2 

One may check that, in order for a term 

S^j g) • • • (g S/J2m-3-fc 

(k) . . . . fVi 

in Pr to contribute to Frug , the partition /i^ must be empty if the i rectangle 
in the k + row of the rectangle diagram is empty, while it must have length at 
most one unless the i*'' rectangle is the leftmost non-empty rectangle in the k + l*** 
row. To be precise, the term s^^ g) • • • (g s^^^^g^j. contributes to only if /i^ has 
length at most one for i =/= m — 1 — k, and is empty when i < m — 2 — fc and when 
i > m. The reason is that all rectangles in the diagram have height at most one, 
which means that any in the algorithm can have length at most one. So if any 
fii has two or more rows, boxes are forced to the right, creating a new partition 
with too many rows ifi^^m — 1 — fc. 

An examination of the algorithm then shows that Pr'''' contains only one such 
term, with coefficient 1. This term has /im-i-fc equal to the staircase partition 
with m — 1 — fc rows, /im-i-fc = (™ ~ ^ ~ k, . . . ,2,1), while any other non-empty 
partitions fii is a single row with m — 1 ~ k boxes, fii ~ (m — 1 — fc). 

Taking fc = 0, we see that F-^j^ = S(m-i,m-2,....2,i)- This was first proved by 
Stanley , and implies that the number of reduced words for wq is equal to the 
number of standard tableaux on the staircase partition with m — 1 rows. 

5. Redundant rank conditions and products of permutations 

Suppose we are given a sequence of bundles Eq —>■ Ei En and a set 

of rank conditions r = } for this sequence. The degeneracy locus nr{E,) is 
then the subset of points x ^ X over which the maps on fibers satisfy all of the 
inequalities 

rank(£'i(x) — > Ej{x)) < Vij 

for i < j. Some of these inequalities may be redundant in the sense that they follow 
from other inequalities. It is easy to see that the inequality involving the number 
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rij is redundant if and only if this number is equal to one of rij^i or In 
other words, the inequality involving is necessary if and only the rectangle Rij 
is not empty. 

Now suppose there are integers < p < q < n such that the rectangle Rij 
is empty whenever exactly one of i and j is in the interval [p, q] . In this case 
the degeneracy locus ^r{E,) is the (scheme-theoretic) intersection of two larger 
loci Qr'iE',) and for the sequences E'^ : Ep ^ ■ ■ ■ ^ Eq and 

E',' : El ■ ■ ■ Ep^i Eq^i ■ ■ ■ ^ En, where r' and r" are the restrictions 
of the rank conditions r = {vij} to these sequences. We will say that E'^ is an 
independent subsequence. Note that if p = g, the bundle Ep is redundant and can 
be removed from the sequence E, without changing Qr{E,). This special case was 
described in Q. 

When E'^ is an independent subsequence, the rectangle diagram for the rank 
conditions r' simply consists of the rectangles Rij for p < i < j < q, while the 
rectangle diagram for r" contains the remaining non-empty rectangles. 

Rectangle diagram for r Rectangle diagram for r' 

m m I □ □ — □ □ 

m \ \ n - - m □ 



Rectangle diagram for r" 



This in particular means that the expected codimensions of flr'{E'^) and flr"{E'J) 
add up to that of nr{E,). If all of these loci have their expected codimensions, 
then we get the equality [Or.(£'.)] = [rij.'(i?^)] • [^r"{E',')] in the cohomology ring of 
X. To see this, note at first that both fir'(-E^) and Ur"{E'J) are Cohen-Macaulay 
12| (see also Lemma A. 2]). If / : flr'iE'^) ^ X is the inclusion, we therefore 



get 

mE.)] = h[r\^r"{E':))] = unnr,.{E':)] = [nAK)] ■ [n^^iE:)] . 

This means that the formula Pr satisfies 

(6) P,. = ( 1 ■ ■ ■ l (E)Pr' 1 ■ ■ ■ ^ 1 ) ■ $g-P+^(fi») 

p n — q 

where multiplication is performed factor-wise, and $p denotes the fc-fold coproduct 
expansion of the p*^ factor of its arguments, i.e. 



We will apply this to study the Schubert polynomial of a product of two permu- 
tations. If w G Sm and u € Sn are permutations, define the product w xu € Sm+n 
to be the permutation which maps i to w{i) if 1 < i < m, while m -|- i is mapped 
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to TO + u(i) for 1 < i < n. The rank diagram for this permutation is equal to 
that of 1™ X u, except the bottom 2m — 2 rows are replaced by the rank diagram 
for w. The diamond of non-empty rectangles in the rectangle diagram for wxu 
is therefore split into a top part containing the diamond of rectangles for u and a 
bottom part with the diamond of rectangles for w. 




Given a sequence of bundles consisting of a full flag of length m + n — 1 followed by 
a full dual flag of the same length as in Section we deduce that the locus flwxu 
is the intersection of the loci fi^ and fiin. xu- We therefore recover the well known 
formula (4.6)] 

C^) ®toxti — ' ©l^xu 

for the Schubert polynomial of a product of two permutations. This immediately 
implies Stanley's identity F^xu = Fw ■ F^ pT| . Note that the same argument shows 
that (Q) also holds for Fulton's universal Schubert polynomials 

6. Relations to a conjectured Littlewood-Richardson rule 

In this flnal section we will discuss relations with Stanley symmetric functions of 
a generalized Littlewood-Richardson rule which is conjectured in We will need 
the notions of (semistandard) Young tableaux and multiplication of tableaux, see 
for example pO| . 

A tableau diagram for a set of rank conditions r — (nj ) is a filling of all boxes 
in the corresponding rectangle diagram with integers, so that each rectangle Rij 
becomes a tableau . Furthermore, it is required that the entries of each tableau 
Tij are strictly larger than the entries in tableaux above Tij in the diagram, within 
45 degree angles. These are the tableaux Tki with i < k < I < j and {k,l) ^ (i, j). 

A factor sequence for a tableau diagram with n rows is a sequence of tableaux 
(Wi, . . . , Wn), which is obtained as follows: li n = 1 then the only factor sequence 
is the sequence (Tgi) containing the only tableau in the diagram. When n > 2, a 
factor sequence is obtained by first constructing a factor sequence (C/i, . . . , Un^i) for 
the bottom n — 1 rows of the tableau diagram, and choosing arbitrary factorizations 
of the tableaux in this sequence: 

Then the sequence 

{Wl,...,Wn) = (Toi • Pi , Ql • Ti2 • P2 , . . . , Qn-1 ' T„_i,„) 
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is a factor sequence for the whole tableau diagram. 

In 1^ it is conjectured that the coefficient c^(r) is equal to the number of different 
factor sequences (Wi, . . . , W„) for any fixed tableau diagram for the rank conditions 
r, such that Wt has shape /z^ for each i. This conjecture has been proved (using 
an involution of Fomin) when all the rectangles in the fourth row of the rectangle 
diagram and below are empty, and no two non-empty rectangles in the third row 
are neighbors Q . 

If r = (fij) are the rank conditions given by a permutation w, then the conjecture 
implies that Stanley's coefficient a^x is equal to the number of different tableaux W 
of shape A', for which (0, . . . , 0, M^, 0, . . . , 0) is a factor sequence. Thus a proof of the 
general conjecture will give a new proof that Stanley's coefficients are non-negative, 
as well as an interesting way to compute them. 

Example 3. Let w = 2 1 4 3 . . . (2p) (2p - 1) e for some p > 0. Then the 
rectangle diagram for w has a 1 x 1 rectangle in the middle of row 4i + 1 for 
< i < p — 1. All other rectangles are empty. 

I I I □ - - - - 



I □ - - 

■ - 

- 

□ 

A tableau diagram is obtained by filling the numbers 1, 2, ... ,p in these boxes. It is 
easy to see that a sequence (0, . . . , 0, 0, . . . , 0) is a factor sequence for this diagram 
if and only if is a standard tableau with p boxes. Therefore the conjecture 
predicts that Stanley's symmetric function is given by 

Xhp 

This can be confirmed using Stanley's formula Fwxu = Fw ■ F„ Let a = 21 G 
S2 ■ Then w = cr x ■ • ■ x cr (p times) , which implies that 

F^ = {F,r = {s^r = j2f'sx. 

Xhp 

We thank F. Sottile for showing us a different proof of this fact. 

Using a criterion for factor sequences given in one may also prove that the 
conjectured Littlewood-Richardson rule gives the correct prediction for Stanley's 
symmetric function of a longest permutation wq. 

In general, Stanley's symmetric function F-^j is known to have a minimal term 
s>(„,) and a maximal term s^((^-), both occurring with coefficient one. If w G S^+i, 
define 

rp{w) = #{(7 \ q < p and w{q) > w{p)} 

for 1 < p < m + 1, and let X(w) be the partition obtained by arranging the numbers 
ri{'w), . . . , rm+i{w) in decreasing order. Let be the conjugate of the partition 
X{w~^). Then a^^j xiw) = <^w,ii{w) — Ij £^nd any partition A with a^x ^ is between 
\{w) and [liyS) in the dominance order p7[ |. 
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Let {Tij}i<i^j<2m be a tableau diagram for (the rank conditions given by) w. 

There arc two extremal ways to form a factor sequence (0, . . . , 0, T4^, 0, . . . , 0) for 
this diagram. The first is to make all factorizations of inductive factor sequences 
{Ui, . . . ,Uk) be "rightward" whenever possible. This means that when factoring Ui 

into Ui = Pi ■ Qi, we take Pi = % and Qi = Ui for i ^ m while we take P„j = Um 
and Qm = (if A: > m). The middle tableau in the final factor sequence then is 

W^right = Tm ' Tm+1 ' • • • • 2l2m-l 

where 

Tj — Toj ■ Tij ■ . . . ■ Tm-i,j ■ 

Note that each tableau Tj has only one column. If we set p = 2m + 1 — j and 
q = ur^{i + 2) then Tij is non-empty if and only if q < p and w{q) > w{p). It 
follows that Tj has exactly rp{w) boxes. 

We claim that Wright has shape A(w)', corresponding to the maximal term of 
. It is enough to show that the if T; and Tj both have a box in row t and I < j, 
then the box in T; is smaller than the one in Tj. To prove this, let the t^^ box in T; 
come from Tki and the t^^ box in Tj come from Tij . If the box in Tki is not smaller 
than the box in Tij then k < i. Now since the tableau Tn must be as wide as Ti~i 
and as tall as , this tableau Tn can't be empty. Similarly, if T^j corresponds to 
a box over T^j in Tj, then Thi gives a corresponding box in T;. This shows that 
the boxes corresponding to Tki and Tij in T; and Tj was not in the same row, a 
contradiction. 

Similarly one can show that the tableau obtained by "leftward" factorizations, 

WlcSi = {To^m ■ To^rn+l ' ' ' ro_2m-l) ' (^l,™ ' ' ' 2^1,2771-1) ' ' ' {Tm-l,m ' ' ' ?m-l,2m-l) , 

has shape /i(w)'. 
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